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MATCHING  JOBS,  PEOPLE,  AND  INSTRUCTIONAL  CONTENT: 

AN  INNOVATIVE  APPLICATION 
OF  A  LATENT  SEMANTIC  ANALYSIS-BASED  TECHNOLOGY 

Introduction.  Modern  organizations  are  increasingly  faced  with  rapid  changes  in  technology 
and  missions,  and  need  constantly  changing  mixes  of  competencies  and  skill.  Assembling 
personnel  with  the  right  knowledge  and  experience  for  a  task  is  especially  difficult  when  there 
are  few  experts,  unfamiliar  devices,  redefined  goals,  and  short  lead  times  for  training  and 
deployment.  When  too  few  adequately  trained  personnel  are  available  for  suddenly  critical 
tasks,  organizations  need  the  ability  (a)  to  identify  existing  personnel  who  could  perform  the 
task  with  the  least  training,  and  (b)  to  create  new  training  courses  quickly  by  assembling 
components  of  old  ones. 

Current  solution  methods  for  such  problems  require  large  investments  of  expert  labor 
and  are  often  either  unacceptably  slow  or  insufficiently  effective.  For  example,  determining 
whether  a  particular  person’s  background  for  a  particular  assignment  requires  some  time- 
consuming  training  component,  or  allows  it  to  be  omitted,  would  probably  necessitate 
intensive  general  training  research  as  well  as  extensive  individual  questioning.  Creating  an 
efficiently  individualized  training  course  would  be  just  as  difficult.  While  a  well- dev  eloped 
engineering  art,  in  the  best  case  course  design,  takes  many  months  of  specialized  analysis  and 
trial. 

Thus,  more  effective  methods  are  desired  for  characterizing,  locating,  and  training 
personnel  who  can  optimally  perform  the  set  of  duties  required  by  any  new  mission. 

This  calls  for  information  technologies  that  can: 

(a)  represent  knowledge  and  skills, 

(b)  identify  people  with  all  or  parts  of  the  knowledge  and  task  experience  required  by 
a  mission — wherever  and  in  whatever  occupation  they  are  currently, 
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(c)  determine  precisely  what,  if  any,  retraining  each  person  needs  in  order  to  perform 
which  new  duties, 

(d)  reduce  the  effort  required  to  create  new  training  programs,  and 

(e)  minimize  the  time  required  for  training  and  retraining. 

The  objective  of  this  research  was  to  develop  and  test  the  practical  capability  of  Latent 
Semantic  Analysis  (LSA),  in  application  to  these  problems.  LSA  is  a  machine- learning 
method  for  automatically  extracting  and  representing  knowledge  in  massive  databases  of 
relevant  electronic  text  (Deerwester,  Dumais,  Furnas,  Landauer,  &  Harshman,  1990).  It  was 
developed  through  ten  years  of  basic  and  applied  research  supported  by  Bell  Communications 
Research  (now  Telcordia),  Defense  Advanced  Research  Projects  Agency  (DARPA),  Office  of 
Naval  Research  (ONR),  Army  Research  Institute  (ARI),  National  Space  and  Aeronautics 
Association  (NASA),  Air  Force  Research  Laboratory  (AFRL),  the  McDonnell  Foundation 
and  others.  LSA  has  been  extensively  validated  in  both  controlled  experiments  and  field  tests 
(Landauer,  1998;  Landauer  &  Dumais,  1997;  Landauer,  Foltz,  &  Laham,  1998).  See  Landauer 
and  Psotka  (2000)  for  a  fuller  explication  and  for  references  to  publications  in  which  details  of 
method  and  validation  can  be  found.  For  demonstrations  of  the  capabilities  we  describe,  see 
http://www.knowledge-technologies.com/HeadHunter/. 

The  new  personnel  data  mining  application  of  LSA  exploits  the  explicit  and  implicit 
knowledge  that  already  exists  in  extensive  textual  computer  files  of  systems  documentation, 
training  and  test  materials,  task  analyses,  and  service  records.  The  current  research  primarily 
used  a  small  subset  of  existing  Air  Force  data  relevant  to  2,121  defined  tasks  performed  in 
three  Air  Force  occupations  by  9,215  airmen.  The  results  showed  that  the  technology  could 
accurately  estimate  the  similarity  of  each  task  or  occupation  to  every  other  task  or  occupation, 
measure  the  degree  of  match  of  each  airman  to  every  task  or  occupation,  estimate  which 
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airmen  could  most  easily  take  the  place  of  others,  and  indicated  that  LSA  has  the  potential  to 
identify  in  detail  and  match  the  knowledge  components  required  by  new  systems  with  those 
contained  in  segments  of  existing  training  materials,  and  with  the  experience  of  individual 
airmen.  The  natural  language  query  design  intrinsic  to  LSA  eliminates  or  alleviates  most  of 
the  known  problems  inherent  in  keyword  matching  of  field-restricted  databases. 

Description  of  LSA  representations.  The  biggest  advantage  of  LSA  knowledge 
representation  for  the  present  purpose  is  that  different  types  of  data  objects  (e.g.,  occupations, 
job  tasks,  personnel,  training  materials)  can  all  exist  as  vectors  within  the  same  semantic 
space  and  can  therefore  be  directly  compared  to  each  other  in  meaningful  ways.  People  can 
easily  make  holistic  judgments  of  similarity  between  a  task  to  be  performed  and  a  set  of 
people  who  might  be  called  upon  to  perform  the  task.  However,  the  structure  in  which  this 
information  is  usually  stored  in  computer  files,  i.e.,  in  relational  databases,  has  precluded  the 
possibility  of  automated  judgments  of  this  sort. 

Traditional  database  structures  are  very  brittle  in  that  search  and  retrieval  are  overly 
dependent  on  specific  data  field  choices  (e.g.,  zip  code  field,  job  title  field)  and  on  exact 
keyword  matching.  While  in  many  cases  exact  matching  on  highly  structured  data  is 
desirable  (e.g.,  find  the  names  of  all  people  in  zip  code  30405),  in  many  other  cases  the 
choices  can  be  overly  restrictive  and/or  ambiguous  (find  all  the  people  who  list  their  job  title 
as  “Doctor”).  In  the  latter  example,  those  people  who  listed  their  title  as  Medical  Doctor, 
Physician,  Surgeon,  General  Practice  Doctor,  and  other  medical  specialties  would  not  match 
the  query  and  would  be  inappropriately  excluded. 

Air  Force  occupational  analysts  have  successfully  employed  a  Task  Inventory 
approach  to  categorize  common  tasks  within  occupational  specialties.  There  are  numerous 
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occupations  within  the  Air  Force — the  Airman  Classification  Structure  Chart  details  the  190 
Specialty  Occupations  currently  active — most  of  which  are  non-combat  and  which  have 
corresponding  civilian  jobs.  Extensive  data  gathered  by  the  Air  Force  Occupational 
Measurement  Squadron  (AFOMS)  is  available  which  describes  the  various  occupations,  at 
least  in  part,  as  lists  of  their  relevant  task  duties.  In  Air  Force  Occupational  Surveys, 
personnel  are  directed  to  estimate  their  time  spent  on  these  various  tasks.  These  Occupational 
Surveys  served  as  the  primary  source  of  data  for  the  research. 

Software  programs  were  developed  to  quickly  create  simple  text  representations  from 
the  existing  AFOMS  databases  for  different  occupations,  tasks,  and  personnel.  For  example, 
the  software  creates  an  Airman  Object  by  gathering  selected  service  status  data  and  the 
descriptions  of  tasks  each  airman  specified  as  having  performed.  The  airman,  from  FSA’s 
point- of- view,  is  represented  as  the  list  of  tasks  performed  by  the  airman  (and  other  selected 
service  status  items).  Additional  relevant  information,  such  as  a  self-description  of  ability,  a 
resume,  or  the  training  courses  the  person  had  passed,  which  would  make  the  description  of 
the  person  more  complete  and  robust,  could  easily  be  added  to  the  personnel  representation  if 
available. 

The  data  from  three  Air  Force  Specialty  Codes  (AFSC),  Aerospace  Mechanics 
(2A5X1),  Physical  Therapists  (4J0X2),  and  Medical  Services  (4N0X1)  were  used.  A 
semantic  space  was  developed  which  included  20,000  documents,  or  object  representations. 
The  objects  were  classified  as  occupations  (full  set  of  tasks  for  each  AFSC;  N  =  3),  duty  lists 
(tasks  grouped  into  functional  units;  N  =  44),  tasks  (individual  task  units;  A  =  2,121)  and 
airmen  (task  and  case  record  data;  N  =  9,215).  Additional  randomly  selected  general 
knowledge  documents  ( N  =  8,617)  not  related  to  the  Air  Force  data,  were  included  to  provide 
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additional  examples  of  words  used  in  context  so  that  LSA  could  create  a  more  robust  space 
with  a  more  extensive  vocabulary.  The  total  number  of  objects  in  the  system  is  20,000 — a 
very  small  dataset  compared  to  LSA’s  capabilities,  but  enough  to  do  a  fair  job  in  estimating 
statistical  regularities  for  the  three  occupations. 

By  adding  less  than  9,000  general  reading  documents,  the  semantic  representations  of 
more  than  50,000  additional  words  were  included  in  the  space.  This  additional 
representational  power  allows  user  synonyms  to  match  task  analysis  keywords.  For  example, 
the  space  would  consider  tire  to  be  a  synonym  for  wheel ,  or  physician  a  synonym  for  doctor 
even  if  tire  and  physician  were  never  used  in  the  task  analysis  data. 

Figure  1  shows  a  two-dimensional  (2-D)  representation  of  two  LSA  objects,  a  Job  and 
a  Candidate.  In  actual  LSA  representations,  it  requires  100-500  orthogonal  dimensions  to 
characterize  an  object — this  2-D  representation  is  for  illustrative  purposes  only.  Unlike 
traditional  factor  analysis,  where  the  dimensions  have  been  interpreted  and  named,  LSA 
dimensions  are  not  in  and  of  themselves  meaningful.  Both  objects  are  seen  as  points  in  the 
Semantic  Space  having  some  score  on  the  X  dimension  and  an  independent  score  on  the  Y 
dimension.  In  most  of  the  work  reported  in  this  paper,  each  of  the  objects  has  scores  for  300 
orthogonal  dimensions,  thus  each  object  is  represented  as  a  vector  of  300  numbers,  rather  than 
a  vector  of  two  numbers.  In  actual  LSA,  the  meaning  of  an  object  is  determined  when  the  full 
set  of  dimension  scores  is  used  in  the  comparison  of  the  object  to  other  objects.  In  most  cases 
LSA  uses  the  cosine  of  the  angle  between  objects  as  the  measure  of  similarity. 
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Figure  1.  A  2-D  representation  of  two  LSA  objects.  In  this  representation,  it  is  seen  how 
quite  different  objects,  here  a  description  of  a  Job  and  a  description  of  a  Candidate  for  the  Job, 
are  cast  in  the  same  semantic  space  in  LSA.  Both  objects  are  seen  as  having  some  score  on 
both  dimensions.  In  actual  LSA  Semantic  Spaces,  objects  are  represented  using  from  100-500 
dimension  scores. 

In  Figure  2  the  capacity  to  make  similarity  comparisons  between  different  data  objects 
is  illustrated.  To  determine  which  of  two  candidates  is  best  suited  for  a  job,  the  LSA  system 
would  use  the  smaller  of  the  two  angles  1  and  2.  Cosines  range  between  1  and  - 1  for  all 
possible  angles — an  angle  of  0  degrees  has  a  cosine  of  1  (the  vectors  lie  on  top  of  each  other), 
an  angle  of  90  degrees  has  a  cosine  of  0  and  an  angle  of  180  degrees  has  a  cosine  of  - 1.  The 
smaller  the  angle,  the  higher  the  cosine,  and  the  more  similar  the  two  objects  are  considered. 

In  the  Figure  2  case,  Candidate  1  is  a  better  choice  for  the  Job. 
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X  Dimensio  n 

Figure  2.  Two  candidates  for  a  job.  In  LSA  any  point  in  space  can  be  compared  to  any  other 
point,  thus  disparate  types  of  objects  can  be  transparently  compared.  Using  the  cosine  of  the 
angle  between  vectors,  we  can  compare  both  Candidates  to  the  Job.  In  this  2-D  illustration,  it 
can  easily  be  seen  that  Candidate  1  is  more  like  the  Job  than  is  Candidate  2.  All  else  being 
equal,  LSA  would  suggest  that  Candidate  1  is  the  better  match  for  the  Job. 

In  Figures  3  and  4,  the  scenario  of  selecting  the  best  training  for  a  Candidate  seeking  a 
particular  Job  is  demonstrated.  In  Figure  3  the  vectors  for  the  Job  and  the  Candidate  are 
displayed  as  before,  along  with  the  vectors  for  two  training  Courses  (A  and  B)  that  have  been 
designated  as  appropriate  for  the  Job.  In  Figure  4  it  is  seen  that  while  both  Courses  contain 
knowledge  important  for  the  Job,  the  appropriate  choice  (Course  A)  as  determined  through 
LSA  would  bring  the  Candidate  much  closer  to  the  Job  vector  than  the  alternative  choice. 

Given  a  larger  set  of  possibly  related  courses  to  choose  from,  the  situation  could  be 
somewhat  more  complicated.  For  example,  given  three  courses,  A,  B,  and  C  containing 
knowledge  a  particular  airman  needed  but  lacked,  A  might  be  at  too  advanced  a  level  for  the 
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individual  to  understand  given  his  or  her  previous  knowledge,  B  might  overlap  too  much  with 


what  the  individual  already  knew  to  be  of  much  value,  and  C  might  be  "just  right",  containing 
just  the  new  knowledge  that  the  person  is  ready  to  learn.  LSA  methods  capable  of  picking  the 
optimum  text  under  these  circumstances  have  also  been  developed  and  empirically 
demonstrated  (Wolfe  et  al.,  1998;  Rehder  et  al.,  1998) 


Figure  3.  A  job,  a  candidate,  and  two  training  courses  for  the  job.  Both  Course  A  and  Course 
B  have  been  identified  as  appropriate  training  for  the  Job,  and  both  are  courses  that  the 
candidate  has  not  yet  taken.  Which  Course  is  most  appropriate  for  the  Candidate  to  take? 
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a  set  of  occupational  specialty  training  courses  based  on  the  content  of  their  final  exam  items. 


We  were  provided  with  text  of  the  multiple- choice  question  and  correct  alternative  answer  for 
each  of  100  items  for  each  of  95  specialty  courses  (AFSC).  For  security,  the  content  was 
encrypted  before  being  sent  by  replacing  each  unique  word  wherever  it  appeared  with  a 
previously  assigned  string  of  random  digits.  From  this  we  constructed  a  matrix  of  -9,500  test 
items  by  -20,000  unique  word- types  and  performed  LSA  at  300  dimensions  to  obtain  a  vector 
representing  each  test  item. 

To  test  how  well  LSA  had  captured  the  semantic  similarities  of  the  domain,  we 
computed  the  similarity,  measured  by  cosine,  of  each  item  with  that  of  100  randomly  chosen 
items  from  the  same  test  ( Within )  and  with  100  randomly  chosen  items  from  all  other  tests 
{Between).  Item  pairs  from  the  same  test  had  higher  average  cosines  over  all  tests  ( n  =  9,500, 
t  =  35.2,  p  <  .0001),  significantly  so  in  82  of  95  of  the  individual  cases  ( n  =  100,  p  <  .05).  See 
Figure  5. 


.03' 

Within  COS  Between  COS 

Figure  5.  Average  cosines  within  and  between  course  test  items 
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We  also  created  an  LSA  vector  for  each  course  by  finding  the  centroid  or  average,  of 


the  100  test  items  for  each  course.  A  subsequent  hierarchical  clustering  of  the  95  centroid 
vectors  for  courses,  done  blind,  produced  groupings  later  judged  to  be  intuitively  meaningful 
and  useful  by  knowledgeable  Air  Force  training  researchers.  The  cluster  analysis  grouped 
courses  for  similar  specialties  together  and  revealed  content  redundancies  between  the  courses 
that  were  useful  in  subsequent  course  restructuring. 

Comparison  of  Airman  Similarity.  In  an  analysis  of  the  semantic  space  developed 
from  the  AFOMS  data  on  9,215  airmen,  the  system’s  estimates  of  similarity  for  airmen  within 
a  specialty  were  compared  to  the  judgments  of  similarity  between  specialties.  The  average 
LSA  similarities  were  calculated  between  3,000  randomly  sampled  pairs  of  airmen  either 
within  an  occupation,  or  between  occupations.  Each  airman  was  compared  with  a  random 
sample  of  others  both  within  their  own  AFSC  and  between  the  alternative  AFSC.  The  LSA 
similarities  of  airmen  within  an  AFSC  were  significantly  greater  (p  <  .0001)  than  the 
similarities  between  airmen  in  different  AFSCs  (see  Table  1  and  Figure  6).  The  specialties 
that  were  more  diverse  showed  less  similarity  for  the  airmen  within  the  specialty  than  did  the 
specialty  that  was  more  uniform.  Subject-matter  experts  informally  concurred  with  the 
judgments  of  similarity  for  a  small  random  sampling  of  the  42,453,505  independent  similarity 
judgments  available  with  a  sample  size  of  9,215  airmen. 

T  able  1.  Similarities  for  occupational  specialties 
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Figure  6.  Mean  cosine  of  personnel  comparisons  within  and  between  occupational  specialties 

Clustering  of  Duty  List  Items.  Another  experiment  looked  at  how  the  representations 
for  the  three  Occupations  and  the  44  Duty  Lists  (tasks  grouped  into  functional  units)  clustered 
in  the  semantic  space.  The  analysis  shows  the  cluster  distribution  of  shared  and  unique  duty 
lists  among  the  occupations.  The  within  specialty  duty  lists  clustered  tightly  around  their 
respective  occupation,  while  those  duties  that  are  shared  across  occupations,  such  as  training, 
management,  and  supervisory  tasks,  clustered  together  and  distinct  from  the  occupations 
regardless  of  originating  specialty  and  differences  in  specific  task  representations. 

Potential  Applications  of  the  Method. 

Job  Placement  or  Occupation  Assignment.  Practical  apphcations  to  job  assignment 
were  most  directly  illustrated  by  the  research  just  reported.  The  simplest  case  is  direct 
replacement  of  one  airman  with  another.  For  this,  a  query  takes  the  form  of  the  to-be- 
replaced  airman’s  identification  number,  and  the  k  most  similar  airmen  known  to  the  system 
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(potentially  all  those  in  the  Air  Force  plus  others  where  relevant)  are  returned  and  listed  in 
terms  of  their  overall  task-experience  pattern — the  closeness  of  their  points  in  the  joint 
semantic  space  representing  tasks,  occupations,  and  airmen.  Their  complete  service  records 
can  then  be  displayed  and  compared.  If  it  is  desired  to  add  a  new  member  to  a  work  group, 
the  descriptions  of  those  tasks  that  are  most  in  need  of  additional  help  can  be  entered  as  the 
query  and  the  system  will  list  in  order  those  airmen  whose  total  experience  is  most  like  the 
new  job  requirements.  Note  that  in  performing  this  match,  LSA  goes  beyond  simply  counting 
the  number  of  tasks  in  common  between  the  wanted  list  and  the  service  record,  instead 
factoring  in  previous  experience  (and,  later,  training)  in  occupations  and  tasks  that  are  similar 
but  not  identical  to  those  in  need  of  performance.  Thus,  it  would  be  quite  possible,  in  the 
absence  of  any  airman  who  had  done  any  of  the  prescribed  tasks,  to  nevertheless  find  one  or 
more  candidates  who  had  done  similar  work,  the  estimate  of  similarity  having  been 
automatically  induced  by  LSA  from  the  entire  corpus  of  data  without  human  intervention. 

The  technique  could  be  used  to  add  people  to  perform  new  jobs,  by  adding  to  the 
query  a  free- form  description  of  the  tasks  involved.  Because  LSA  captures  semantic  and 
conceptual  similarity  of  verbal  expressions,  it  will  correctly  match  ad  hoc  task  descriptions 
with  official  task  definitions  and  job  descriptions.  The  system  can  also  form  a  representation 
of  the  overall  mix  of  tasks  required  by  a  group  by  combining  representations  of  the 
knowledge  possessed  by  all  its  present  members.  In  case  of  downsizing,  the  system  would 
make  it  possible  to  find  a  set  of  personnel  to  transfer  out  of  a  group  that  would  either  leave  it 
most  like  its  previous  composition,  or  desirably  modified,  again  without  relying  on  a  crude 
counting  operation  or  intuition. 
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The  opportunity  and  manner  of  application  for  selecting  airmen  for  missions,  for 
example,  expeditionary  war- fighting  missions  with  unique  challenges,  is  relatively 
straightforward.  Given  a  careful  verbal  description  of  the  mission,  including  all  the  tasks  to 
be  performed,  the  equipment,  weapons,  devices,  procedures,  numbers  of  airmen  needed  in 
each  role,  and  perhaps  even  factors  such  as  locale,  terrain,  and  likely  weather  and  other 
challenges,  the  LSA  matching  technique  would  rank  airmen  for  suitability  to  each  task  on  the 
basis  of  the  totality  of  their  previous  task  and  occupational  experience,  along  with,  if 
available,  relevant  (as  determined  by  LSA)  test  scores  and  performance  ratings. 

Curriculum  Overlap  Analyses.  The  Air  Force  (like  other  military  and  civilian 
organizations)  offers  hundreds  of  specialty  training  courses,  many  of  which  overlap 
substantially  in  content,  many  of  which  may  contain  content  no  longer  relevant  to  tasks 
currently  in  demand,  and  some  which  are  missing  content  made  desirable  by  changes  in 
technology,  missions,  or  staffing.  In  many  cases  it  would  be  desirable  to  combine,  condense, 
or  modify  courses.  Teaching  unnecessary  numbers  of  courses  or  redundant  components  in 
multiple  courses  is  expensive  in  instructional  staff  and  facilities,  and  even  more  expensive  in 
wasted  student  time  and  resources.  Teaching  material  that  is  sub-optimally  matched  to  work 
requirements,  either  by  being  superfluous,  redundant,  or  by  failing  to  equip  airmen  with  the 
best  skill  sets  for  all  the  tasks  it  would  be  desirable  for  them  to  be  able  to  perform,  is  probably 
even  more  expensive  in  the  long  run. 

To  rationalize  the  content  and  organization  of  content  for  multiple  training  programs,  a 
method  is  needed  by  which  the  overlap  in  course  content  can  be  easily  assessed.  Presently 
such  analyses  are  performed,  if  at  all,  by  highly  labor-intensive  efforts  by  subject-matter 
experts  and  training  specialists.  We  have  already  demonstrated  that  LSA  can  do  this  kind  of 


14 


analysis  automatically  to  a  quite  useful  degree.  Our  studies  were  based  on  analysis  only  of 
course  examination  items,  but  appeared  to  give  a  great  deal  of  useful  information  about  course 
overlap. 

LSA  can  also  measure  the  overlap  between  course  content  and  the  full  range  of  tasks 
performed  in  many  different  occupations.  Information  from  such  analyses  will  suggest  where 
the  training  needed  for  different  occupations  overlaps  and  might  be  combined,  where  training 
is  lacking,  point  to  components  that  may  not  actually  be  needed  at  all,  and,  in  some  instances, 
suggest  ways  in  which  occupations  might  be  restructured  to  increase  training  efficiency.  LSA 
methods  will  not  solve  these  problems  completely,  but  we  believe  they  can  offer  highly  useful 
information  for  planners  that  is  currently  unobtainable  or  prohibitively  expensive. 

Just-in-timeTraining  Materials.  In  brief,  the  way  in  which  we  envision  that  LSA 
would  be  employed  in  helping  to  rapidly  create  new  targeted  training  programs  might  be  as 
follows.  The  component  knowledge  needed  and  tasks  to  be  performed  for  a  new  device, 
system,  or  procedure  would  be  carefully  described  by  designers  and  relevant  subject-matter 
experts.  LSA  would  determine  the  degree  of  match  of  each  component  to  a  wide  range  of 
tasks  performed  in  the  Air  Force  and  to  every  paragraph  in  every  possibly  relevant  training  or 
operations  manual.  Tasks  and  paragraphs  would  be  ranked  by  estimated  relevance  to  the  new 
system,  and  the  LSA  similarity  of  each  paragraph  to  each  task  determined.  In  the  quickest 
and  dirtiest  version,  a  custom  retraining  document  for  each  candidate  could  initially  be 
compiled  from  paragraphs  highly  relevant  to  the  new  system  that  are  not  highly  similar  to 
tasks  the  candidate  has  previously  performed.  In  the  case  of  urgent  need  for  a  small  number 
of  trainees,  a  subject-matter  or  training  expert  could  then  edit  each  version.  In  case  of  need 
for  large  numbers  and  more  available  time,  the  collection  of  paragraphs  could  be  crafted  into 
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a  simple  computer-based  training  program  with  branching  to  permit  trainees  to  skip  parts  they 
already  know. 

HeadHunter — A  WWW-based  Intelligent  Search  Agent.  The  HeadHunter  software 
represents  an  initial  demonstration  of  a  usable  World-Wide  Web-based  Intelligent  Search 
Agent  based  on  the  LSA  technology.  Currently  the  system  has  knowledge  of  the  three  AFSCs 
used  in  the  research,  however,  even  with  its  limited  knowledge  base,  it  demonstrates  the 
necessary  capabilities  to  match  mission  and  job  requirement  statements  with  military 
personnel  and  training  data.  By  measuring  semantic  similarity  of  training  materials  and  tests, 
it  promises  to  facilitate  combining  occupations  based  on  core  competencies  and  similar  work 
activities.  It  also  promises  to  help  identify  individuals  qualified  for  work  activities  for  which 
no  current  occupation  exists. 

An  organization  that  has  acquired  a  new  or  revised  system  could  develop  detailed 
descriptions  of  the  activities  required  to  operate  or  to  maintain  the  system,  based  on  system 
requirements  documents,  operations  manuals,  or  provided  by  subject-matter  experts.  Given 
such  descriptions,  and  assuming  an  increase  in  its  knowledge,  HeadHunter  could 
automatically  identify  current  jobs  on  fielded  systems  that  are  similar  in  component  work 
activities  and  in  their  requirements  for  training.  It  would  also  identify  similar  paragraphs  in 
existing  course  materials  and  rank  them  by  probable  relevance  to  work  with  the  new  system. 

New  occupations  could  be  structured  around  these  activities  and  new  sets  of  training 
materials  assembled,  at  least  in  major  part,  from  subsets  of  existing  material.  In  addition, 
individuals  who  work  in  jobs  that  use  subsets  of  the  competencies  and  experience  required 
can  be  identified.  This  might  permit  the  immediate  employment  of  appropriate  personnel  or 
their  more  rapid  and  effective  re- training  for  work  in  support  of  new  systems. 
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In  the  occupational  domain,  this  effort  may  ultimately  produce  a  cost-effective 
capability  to  systematically  mine  occupational  personnel  and  training  databases  to  develop 
new  job  and  training  structures  to  support  a  variety  of  requirements.  This  capability  would 
help  employers  identify  critical  characteristics  and  competencies  associated  with  work 
activities  and  then  to  identify  individuals  who  have  the  requisite  experience  and  competencies 
to  perform  the  identified  work  activities. 

Conclusions.  The  preliminary  investigations  using  three  Air  Force  specialties 
demonstrate  the  potential  of  the  LSA-based  system  for  personnel  and  occupational  analyses. 
The  ability  to  create  holistic  LSA  representations  from  pre-existing  structured  databases  is  an 
important  capability  for  development  of  the  technology  for  practical  applications.  The 
importance  of  using  these  techniques  with  large  databases,  e.g.,  for  the  hundreds  of  thousands 
of  personnel  in  branches  of  the  military,  or  even  the  thousands  of  employees  in  large 
corporations,  should  be  emphasized.  The  LSA  tools  allow  for  analyses  that  have  been 
heretofore  impossible  because  of  the  size  and  complexity  of  the  involved  data. 
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